J 



(IB] 



fcuropSlsehe* P«tootwrt 

Euirop«an Patent Offics 

Offira europten des brevete 




@ Appfication number: 95304419l5 
©Date or fling :23.06L95 



® fWcallon number: 0 689 167 Al 
EUROPEAN PATENT APPUCATION 

® Intel': G08T7/00 



@ Priorfly : 24w0eL94 JP 1427M/M 

® ^ of pubOcatbn of application : 
27.1ZS5 BuNsiin B5/S2 

@ Designated ComnBGling States - 
DEFRGBiTNL 

® SKJ% KABUSHIM KAISHA 

OhfaMoi 

@ Inventor: Kmva(, Tomoaki 
^ Canon IOC. «M, »«hom.^ 
Stibnomaniko 
Olrt»liu, T0I90 (JP) 



@ •n»«9« Pfoe^rtng appMhia and method 

^ Inputted right and left taages of a siBra«u 

ItL^^L^T^ to be used for fcrohg aSe^ 

STSo":^"'. f* object «^p3lS 
^ a pomting device 1-7. At this tme. ff obiects 

ton for clMngftjg obfeds is issued. Aocord- 

^'J J?^?** to input and managepSni 
•teta of each otyect independently 



[■nwrior: Okutomi MBtaluM 
30* »chomo, 

Snbnomaniko 
OM»-ki^ Td^ (JP) 

Snbnomaniko 
pM»*n ToigfD (JP) 

("vmtor: Maaaksu 
cto Canon KJC. 30* Schome 
SMnomaniko 
Olrt»*u, T0I90 (JP) 

@ RegBsenlativo : Boierttort. D^^^ 

BBIESFORO ft Col 
2-a Wanvick Court 
HIgtiHolbom 
London WCIR 5DJ (6B) 



18. lue SahMtonh. 75001 PAWS 




EP0689167A1 

BACKGROUND OF THE INVEhmOM 
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Ihe present VYvention relates to an tmaoe Drocessmn nnr^orr^.^ «^ ^ ^ 

image p««ssh9 apparatus and meth^ZSSr^Sfir^ 
eoscopic image using rt9M anSlSmS^^ 

andlhellke. '^""''^''"^'^^•^noM and left to 

al«»com«^potoS::SSS;^ "-Buse 
consWenAle tin» 10 obWn the bS^*^^ 

*br»Wln9at..ZSSL::^^ 
1993). According to thbmetlKxl rf^tl»M«r!fiI!^Z^ 

points b generalBdL tl«fbni>lnQ a Dto™^^ 
SUMMARY OF THE INVEWnOM 



of 8 stereoscopN: image consisting of one or more objects, designates oarsof »J^^^ 

and left in««^ se<nientiaB,. and extracts thn^Sfi^C.^^.^^^:^ 

means for drawing a supplementary line or lines in the displayed 'i^hTZXr^^^J^J^'^ 

f~g«««^^pointsr«n.^r,.andler.i3ont^JZ?^ 

"'^^™**^«»™*™<»'tt»P^tin»entiontopmyjdeanima9epn>^ 
^^<^env^„^,.extracting..«edimens^^ 
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AcconflngV. there bprovWed an lmaQeimeBsrf»«».^ ^ 

JnptmBd rigw and left rn«9es o# a 8Ib3^SSSS^ 

^of IfH, rIgMand left .maoes: and^SSSt^^^ 
lBr»«c pc*^h tha ollMr Image on theS^^^ 

"aw •Vll»nrstdes*B„aHon mea,«ZSr^ 

A8wiabedBscribedl«to^.accordih9toC^^S^ 

8»nal 8l«ve date by ewilching tt»lKto to 

P«*«^«» texture Wbrmati^ «*«^bSS?,f5?aSS?^ ""JSnftwl^ 

Further, a function which auiomaticanv etwZiTZ 

Further. bydispiaySngnartoWlcating^S^^ 
•S^J- P<-nts. the positions of the 

8po«l»^ pointe can l» ca«»led and3«iCSiS^^ 

BypreyidSngamethodlbrdelemrfninflaMlS^ 
pa^ofoonespondln8point8andX3»Sri^T!^'^i^ 
.n an areawhere the texture Irtbnnatlon is unS^J^ 

f one of aforesaid n^thod is used in on*^ to ^'^S^^S^^'^ 
>''^orcurvedllnes)l„therigWand.eftlnw9es«ZL^iT^^^^ 
suchasintersecfionsof.hesupp.en«ntan^^«^r^^ 

Other feahires and advantages of the i)reserti~«^T^ 
enjn conjunction With accSSnyJ^T^^ 

similar parts throughout the figures th^. ^^ft^nce characters designate the same or 

BRIEF DESCRIPTIOW OF THF nPAWiM^o 

Fig. 1 is a blo<* diagram showiTa coC^^aJrJTrj^"'*"'^^^ 

F«. 2 shows program modules acxordingTS^iS cZSS^ ^ embodiment; 

Fig. 3»adlagram ilustrating states of Zjla^riS^^ 

stage accortlngtothe first embodiment; 

s^^'^oscopic image at an initial 

Fig. 6 is a diagrem showirS a const^^ 2 a SS^S^^"^ ''^ 

Fig. 7 is a table showing^r,^^^ "^^^^^ ^"^"^ »»» embodiment; 

the first embodiment; "^"^^ «««9 names, the point list, and the^. list used in 

Fig. ^»*«a««-hartsho.^aZSsto'^^:.S^^ 

wrea according to the f irst embodiment 
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<Rret Embodtment> 

««scnoea alx««. data of ngw and left Images of a stereos««!n • 

^ w a stereoscopic image obtained 
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lVPfc«nguj>byastereosoopfcc»neia.forexai»te teHi^w^u. 

devfce is competed to the liJ^S^^, m^JJS^S^ 
s««««»«>Pfc Image (simply referred 

numeral 1-21 in Rg. 1). BydesiBnatinglvthetei^tS^TfS^'^""'^ 
andj^lmagesareloadedfremthe^^.^'j;^ t-A ll» desisnated right 

on the display 1-5. Note Itet a method Ibr mpj^ JTSSS^^ 
Pa™|«sof.hepresemim«miD„«„o.llmlW 

canbeinputtedfremafloppydlskdHveoonnecledSSeS^S^ 
aoom™nfcatlonBne.7hehanfdlsk1-28to«^TCi^^ 

«»essho^heachfla^whichw.l1rS^^S^^ 
Praflraro are loaded In the mah Storage mwSnl^ 

r«. 38ho.«anexampletol»^te^!r^ 
nun««d^1 *--es^^X-io!r„;^ -n Fl» 3. reference 

«««Pfc image. fespecUvety; and »4 a wl«fci» fO« and left images of a sfer- 

exeXs::^:^^^^^ 

tt«e pomtfng device 1-7. a list of seteofSw a^^fm^JZ^^!!^ the menu by the keyboart 1-6 or 
h«J «^ 1-2 is displayed. Then, one i^^T^ 

nght and left images 1-21) are loaded in a n^^^^lT^^*^^ the 
anddlsptayedonthedisplihaSS^SZ?^^-^^ mah etorege medium 
togelhar. Aceonlhgiy. «« feft and fiMi^^L^^'^''^.'^^ 

and 3-3. respec«vety. on the ^^f^^X^n^^^^^S^jT^JT ^^"^ " •in.SeS 
display1-5. '^^ ^ s««»<wm 

the poirtlngdevice 1-7 on the displayed riflWand^iZI^H^^'^*^ 

i-9esr..«SM:iri?er.!::^^^ 

and left image, are inputted. The designateCTs « ^^^^S^ji^:^^'^ ^ «»» right 
after being applied with successiveT23w!^|"^S^.!^^ dimensional sha^ 

Note that, in the first embodimwrt^^^r!!- coordinates calculation. 

n«nu^lyon.hebasisofs^:r:t::t.tfe^rc^^ 

manghtor left image isdesignated. the coaS^nJS^^"'^^ 

pattern matching and the like. ""^^Ponang po.nt in the other image is automatfcally calcufated by 

sterer.rn::;s::S'r:ri-^ 

designated corresponding poi«s in the ri^^t '^^r^l" ^^^^ ^ "^^^ 
cally. me a>o,di„ate date includes co^^^'^l structure described in r» 5. More specif i- 

samein the left image, three dimensional ^^^^S^^" «f the 

scnbed later on the basis of the coordinates rtTZTa^r^ hj^porformmg cataialtons which wM be de- 
the ne^ poin, and these date are regist^^'in":^'^^"^;^^ 

raspond«,g poims are designated, coordinate deteof^^rfe ^JT'^' ^ « P«^ "ew cor- 

'"f"* 5. regarding an i^h added poht.coordi,«I^o^ri!^^^ 
•mage (Rxi. Ryi). three dimensfenal coonfinaS^^ k ^iS^"^^ ^' «»«""ates on the right 
later on ba^ of the coordinates - -esJL 

Theaforesa«threedimens^^^^,3^-^^W^-^ 
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(deserted be.o«,,SSr3^ 
S425). Note. «» can»n. pa«n«l««tod^ 

Cameia iwrametefs far the li« imaqe and^ 

a™ fl^ In the system l,y any ^ 

Next a plane Is generated on the basis of thB Dotal 

triangutar patcl».a»9««a^ 
^*»«l>»anes.l^specrS^^ZS^ 

VVhen the ffcst lh«eS^.£2^^ 
?«»rttjelhW point. wheneveranewSJ^^ 

"ry*«^P«*-a»8e.ected«,tfJ^ 
» 8*«8«««WearlierVVhenapoWisaddedLtl»I2^ 
everllsnecessarytodoso,so^allSSte<^^ 
13 l8ane«anipteof aDfc<»««.ifc!S \^ 

i3..Hghtwindo:s:j^';?^S^^ 

» wm«k»f13^l8de8aa»dbetow,. ^^^»«« In a ttweedlnienslonal space 

For example, fkst. attention b paW to a trianolB K. 
5^1). (RX2. Ry2). and (RX3. Ry3X 

T»H» triangles for tto left in«ge and h lt» tl^Md^;ZS^ 

«»-*~ns*«*spaoe^«^ triangtes h the 

d.men8lonal8paoewindn».»b„ee«BMryto8eta«^l3i2!^ 
Asn»thodsfargenenrtlngabfaS^SSlT^^l^^ 
"««^Ptene)a,e'S:ZSn«^ - an In^ge (in a two dl- 

the ruJes In the latter method, then, nowth^J^n^wTS^ RegwUng 
Of the Debunays prbCpte. Note ^^^t^^^^^Z^ST'^ 'Z a-tomatfcl^^ 
nays principal, triangutetion netconnoctS ao33«*7.?^ " net generation based on the Delau- 
angle out of al the vertloes of the J^SSl tSZ.^ ^ 
feasdoseasa,|gh.triang,eXrb^S^::^JS^^ 

ever a«a an the pn«esses far generating theXZStaT^ S^^^Jl!^*'^' automatlcaly. Hov*. 
toexplicaiyleayeallneconnectlngtwo^s -n^^^^JZ^^^^ "''"'''"^ 
1^:^, ^ "-S-^CitSri^* has praposed a method 

.^s^tereT^C:--:;^ 

•owing the plane list ^ ^^"^^ *n»ns»nal triangular patches l>y fol- 

-ispJ^tTnSrS^^ 

d«play method, and any three dimens«S^Z,Z^!^^ 
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In onto to overcome the above DToNfim. eftnh frh^ 
each to« I„ tt» a„ 

pn»sses oorraspom^ to 

a case whem ^BfifatBT olrfBii- hi th-i 
&1 Fig. 9 as an example. ™*'^'''*»»~«»»Perti»nwla»sl^ 

« under edIBon (.efa«, as nop 

•**til'^'-tontoafBeof,^andWlSi^ 
» .^-•-Oeo.^.se.ec^.a^rsar^:^ 

» RWiH>ol^ectde8i9natton.alfc*35S8w^ 
desigajtes one Of tt» ol^ects In fte bJ?^^ 

«-ren«y pnK»sse<t has not been regWeSi U^ff iS^S^ «f the ofcjec* which Is 

oJie^laofttmol^ectbdeteedlZrifSiteS^ 

Afhr Changing the polntBri,tt«otieclwh5JS3rt^^ 
u«r«„conffcmlhelmafles(sl^ri^ 

wliKtow(stBpS706). ^^^"®"****'*»<''»P'»)«^'n«ftethreedimenslDnal^^^ 
Whereas, when "save obtecT is selednH in »k» ^.^ 

step S414 to Step S415. pnK««es a S^^^ 

lnlhepn)cessos.dateirfih«n«i«.ii». J^l"™^ "*^^*"P**««™ed: 

.•n.heha„,':Sr?Sl;??te '^"''^'^ 

the list data designated by the^ i^L^ i^l^^^l^^ «» We shown h Fig. 7. Afterv^^te 

Further, when >ead ol^ecr ls«le^^^^!^!t!! 
Shown in a f ^.wchart in P^a^SSS::;:^ 

•nafilebread. 'ist and the plane list stored 

Therr:::;r:::s--<^,^>-^^^ 

(step S904). Thereafter, the object IsVlspl^^Ij^ JfTTJ^ «rf an object Is added to entries under edition 
and^ inthe threedln««*.S^ispS^Sp^!r^ 

point list read In this process. ' ^ ''^ "'^Pow'ing points can be added to the 

When "corfirm disolav" k coww ~- »k„ 
Shown in a flowchartti^il ^^St^.n'^;:^^:^^ '^.'^ step S417. processes 

•stered objects. A phiraliy of ot^s^nJ^feL^^ ^'^ ^n*"*^ M-e reg- 

c^j^^.eraur,:^2,tr;^-«~ 
wHet.«rr::r^;r:^ii^^^ 

name of the additional obie^kl>signJ^^^,1SSl^ » «here is. an ot>jert 

the plane list are fetchedlndset SS^f^^^^^^^J**^ '^'"'^ ft>r the point 
Slice, and also to the three dininsJJld^^:^^^^^^^ 
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» inputted separately. B^STftTJSte^S 
rtgMandlefllm^ayttni^3^|;^'*^ 

playlnlt«edlff»«rfoneyenjfthet8xtuiebZ^te^S^ 

obMisseanrfMrfasaoroupof apluStlrfoSS^ Especfally. when a shape of an 

Hme. However. tCZSLthSsT^SS^al 2^^^^^ '° fast at a present 

«n«edtotrten9le. butcan berectaSe cJ^^^ 

p^'r^CJir:^rss:^ 

dimenstonal coonlihates whfch stbSJ^ SLStS^nlL^lSirSr' ''^ """9 

netbyusi„gpoinlsorthmedmH«*«a»^S2r^fe^ '^"^^ 
•ng device for two or three d!n»nsjons. B^mTZ" designate points manually by a point- 

whW, perns having ctosest dS^ S^^lnt:,;^^!;:!^^ - 
spa«. and combination of above two. J^^Z^^^^^^^^T^ into three dimensional 

Further, not only three dlmensbn;i^S^a^^t^^Sr~ ^^L"""^ 
neously. Therefore, it is advanteoeous to^^^T' ^ "nfomMtion is inputted simulta- 

sional display means in thT^^i^^?!;;^*'^^ '«^- dfncn- 

andleftimagesastexture '''*'^'***^'^P'^»V*««i^'^Bftheronoofthori^^^ 



<Second Em6odimont> 
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5. By swachhfl among the albresafci oDBrrtlnn« n# * 
^«*««fin9 poirts in riiW and 

embodiment Fig. 22 b a^taTSS^^tj^^J'*^^ 

storage mediun, 105 (consSsterfR^JuT^^ «teta of tt» Jn«Ls b IdS;^?^ 

mode.lsdlspteyedonthodisptay 101. ^^Ta^rL^^^ pdnt cooectton mode, anda mmS 
orapoirtlngdeyfce102. ^ '^'^ -"odes by us&Tate^^^ 

dimensional geometrical shape model is JSTiT^ generated. Data of the generated ihn^ 

ang^p^oftheshaZTL^tS^r^,^^ 

When It B instructed that some of the detoZL^^^ ^' <«sp'ay 101. 

Next, contents of step S103 in Fm. ?i »«ii k- ^ 
S301 mFlg.za.apolntVbeave^Sai^^n^::^'''''"^ 23. Rrst. etsteo 
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;^ihtt«lmaflebandabuttooonlhepofc^^ 

T^•poJntde8^B«rtBda»8tepW 
Aandftaa«detonninedasoo„«VonSS^^^ 

f ft the piocess proceeds to step S310 mii>nnMfa.M^i . 

te IV opeiatlng «» pointing deylco 1 02. a TO 
«;;aW^"8in9thBpoWlngdeW^ 

b*i^atalattlcepolntananfleddisc«leV.«w^^^ 
betow.whe„apoin,l„m.tefti:^r7pS:St^S^^^^^ 

0-=..C.B ^ ...(1) 



I = 1 



r = 



^ o=«.e.B..i;i:t - . (2) 



0-».C.B «el ... (3) 
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8«Bp S402 and S403 « «veated un« t ?d3^ 

<Thnrd Embodiment 

Athtrd embodiment win be described Mm, 

In the first and second embodin»nls as descrft)edabaw« n uiHiH'^j**^^^ . 

can be used as the .MrS^^^S^ZTr^^^ "^"^ 

Tl»n. me mpui^, a«,Tit^'alTp.S^S 

obteimng «»ee dln«„«o„a. Jl^aTwrS^ 

to be seen from an arMrary viewpoint is oJ^^ ««T ^^^^ 

displayed on a dfep^zHX^TjnS^^^ ''^ "'''^ 

t ransferred to an^ ^ shape » st«B i„ a file and can be 

keyboard and a pointino de^ ^^-^Z^^HT^^ ^ ««h as a 
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(step S36). the«lv ite^can be ad£ eSS^^J^ 
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POWP4 fe obtained as the intersection beti.»BBn iL^^^^^-fT^ supplementary ikies, and the 
images. Asaresu^dataorS^^p^^^^S:^^ 

expressed wHh dot^ Mn^the fl^^l^^^"*^ Pi P3. P4. and P5. and 

user interfece. Further, intersections among the suppS^SS^nSl^iS^r^ ^ 
nally irxduded in the right and left Im^esSd int^STS!^ 

responding points. Accordingly it b^SSte^STi!^!^ supplementary lines are used as cor- 
Pattern o^iJe pten^T^S^i^^^^^^^^ «> 
•^^-^^-^-^PoScant;':^^ 

T^^erefdre. on the tesis of thus obtaned rinh» i>»h i^f • *2^»«*ponoing points. 

posed U, be seen fromadiffer^C::?^^^ 

thro;;"^:rj"C^ - passes 

-i- <.n be drawn'by S^^tr::2C-^s": ^^^T ^^^^^.^^-^ "^^-"^ 

use th«n ,n ooler to reduce difficuWes and pmcessing tlJ^ 

rt^ns o, ocduded points can be de.en.ined'^!^ arc T.:^^^^^-::::::^^ ^ 
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anipte of triangular patches of a tlwBe dJmensfcwZ^ 

from that In Fi9. 31. 8**''**™=* •'"Pe "wW seen f«m 

Eml»ocDinent> 

ttetlnRg.33. ^^^*»*^*^'*«P*'«^««en»romavta»po{rtdJ^^ 
<Sxlh &nbodfmofrt> 

and S107 in Rg. 21. respectivelK « steps S101. S102. S103. SIM. S109. S105, S106. S111. 

a^tJl^^s^e'l^SSSSS^^ 

not to l« used. tf» corresponding pSrZ^d^S^^' 

<Seventh Embodiment 

uaflyatstepsaoi inFig. 23. ^ '*^'^'^'^«''^""«n9comsspondingpoints\^ 



^Eighth Embodiment> 



atstepS301i„Fi9.23.andtheendofSiXS^te^^^i^S!!^"^''^ 
« J'«teadorthepolntde,ignatedn«„uar«^^a^^'^^ 

<Ninth Embodiment 
<Tenth Embodiment 

« a. ^T^"^ "^^^^ « "-^•-••"9 pen. sea^ n»de and a n»„ua, nnxte 
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<Eleveiitti EmbodIn)errt> 

5 

-«Twemh Embodiment 

10 <niirteenth Embodiment 

As-cor«spondh«poWdetenn*»^ 

<Fourteenth Embodiment 

15 

*'"'^""™'"»ractensticsofima9eslhantt»d^ 

» <Rfteentti Embodlnien> 
<Sixteenth Embodiment 

«s is^i;;^ «»»^^ 

for displaying them. "*"^*""'^''y°^®tored three dimension^ shapes and 

The preset im^rtion «^ ap^Stot c^^^^ TT^ 
ven^«battei„edbysupp,yS„gaprog«mto 

can •""'^^ «^ ---^ona. data f„^ „e Hght a«, .eft l„«ges 

The present invenUon is Bmlled to the above e..«,in«nts and vario. ^ 



45 



50 
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2S 



5 aaims 

and wherein surface of each Akicw^ ;^ 
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to 
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of each object to be dtsptayed ^°^*'^'«*«rf«««o'threediinensfc»J^ 

<«"entttly. and extmcffng ttweZSZ^*^ cSSSL**^" '"."^ rtQht and left Images 
andJl^rr^--^^-^--^^ 

1». ^!™9«Pn«esslngmett»daoc«dln9toclalm18.wti^ 

atMMlsupplemertarylinedrawingste^ ^♦ra'sW " «»irved line or lines are draw^ 

20. The Image processing method accordino to claim ia 
wherelnthreedimenslonalcLrii^r?^^ 
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"•^spondstDtheciwracterisfc 
among ccrreteHonvah^calcS^ 

2i Tl»in««Bp«^^ 

f5^'~"~*^'''''*^'^'Wrt and left hr^ 

near .he c^J^^^^S^^::::^:^ beW^ a first ^ich is Seated 

area andthe 

and whemin said second extracting moans includes: ^ "»ans. 

-O^ccnes^^:?:^--^^ 

near the char^^^L^^^^f^S:!^ first area wh«, is located 

^»«r.«oHertin^SC^SSi~,^"^'^"«^ 

ondde^„atrnrarjs:r^j::,^tre:s^^ 

est anK^gcoc^ation values calcutetedt,ysa«!::.^m^^^ 

.he t^sis Of s::r^"^3s:s^:^ ''^•^ ^ 
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acBsptoyfcigirtBpofdlsptayitigiterightandtefllmag^ * 

3«. ■^«Wl««essfngnrtlwlacconHn9toelalm28.wh^ 
corn*-looyaluescalcul2;?^^ 
3Z The in»gBprocesslnBincttx)d according to daImM 

a selectwn step setect«9 e-her one of saw f irst. se^ 
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and said ftst«tnKil„g step hS^^ 



'«9MorW, image .nwtJlSXBcS^ 

amo„gcom-a«a„ value. cSSS^ST^^ ««- «^e se««d a«a I, g^alest 

first desJftnatlon step. ^•'~™'*^P«^'n"»ott>er image desiftnatBd at saW 
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^ J -^S702 
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I TOR POINT AN D WANE USTS K-S703 
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I WSPLAY OBJECT IN 1 
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